Skip to content

unified: Swift grammar cleanup part 1#21821

Merged
tausbn merged 24 commits into
mainfrom
tausbn/unified-swift-grammar-cleanup-phase-1
May 12, 2026
Merged

unified: Swift grammar cleanup part 1#21821
tausbn merged 24 commits into
mainfrom
tausbn/unified-swift-grammar-cleanup-phase-1

Conversation

@tausbn
Copy link
Copy Markdown
Contributor

@tausbn tausbn commented May 8, 2026

A lot of commits, but most of the churn is just files being regenerated. This only happens in separate commits, so those commits do not need to be reviewed.

@tausbn tausbn added the no-change-note-required This PR does not need a change note label May 8, 2026

/** Gets the node corresponding to the field `element`. */
final AstNode getElement(int i) { swift_array_literal_element(this, i, result) }
final Expression getElement(int i) { swift_array_literal_element(this, i, result) }

/** Gets the node corresponding to the field `raw_value`. */
final AstNode getRawValue(int i) { swift_enum_entry_raw_value(this, i, result) }
final Expression getRawValue(int i) { swift_enum_entry_raw_value(this, i, result) }

/** Gets the node corresponding to the field `default_value`. */
final AstNode getDefaultValue(int i) {
final Expression getDefaultValue(int i) {

/** Gets the node corresponding to the field `name`. */
final AstNode getName(int i) { swift_guard_statement_name(this, i, result) }
final IfCondition getCondition(int i) { swift_guard_statement_condition(this, i, result) }
final override string getAPrimaryQlClass() { result = "IfStatement" }

/** Gets the node corresponding to the field `condition`. */
final IfCondition getCondition(int i) { swift_if_statement_condition(this, i, result) }

/** Gets the node corresponding to the field `default_value`. */
final AstNode getDefaultValue(int i) { swift_macro_declaration_default_value(this, i, result) }
final Expression getDefaultValue(int i) {

/** Gets the node corresponding to the field `default_value`. */
final AstNode getDefaultValue(int i) {
final Expression getDefaultValue(int i) {

/** Gets the node corresponding to the field `name`. */
final AstNode getName(int i) { swift_repeat_while_statement_name(this, i, result) }
final IfCondition getCondition(int i) {

/** Gets the node corresponding to the field `default_value`. */
final AstNode getDefaultValue(int i) {
final Expression getDefaultValue(int i) {

/** Gets the node corresponding to the field `name`. */
final AstNode getName(int i) { swift_while_statement_name(this, i, result) }
final IfCondition getCondition(int i) { swift_while_statement_condition(this, i, result) }
@tausbn tausbn force-pushed the tausbn/unified-swift-grammar-cleanup-phase-1 branch from c972e26 to 7c91dd9 Compare May 12, 2026 12:05
tausbn added 24 commits May 12, 2026 12:57
The astute reader will note that we seem to _lose_ some node types in
the process. Apparently, these were unreachable in the grammar, and the
newer version of tree-sitter removes such "dead code".
This caused any field containing an _expression to appear as if it could
countain any number of such nodes. It also threw away the information
that there was a `?` marker there.

To fix it, we simply move the definition into its own named node.
We make _referenceable_operator a named node. This prevents it from
bleeding through to the _expression definition. It likely also makes the
output easier to deal with, as bare operators used as arguments now have
a named node wrapping them in the AST.

Also removes a duplicated inclusion of _comparison_operator that served
no purpose.
Before, the `condition` field of an if statement supposedly could
contain things like parentheses and commas, due to bleeding from
referenced anonymous nodes. Making the node named makes this issue go
away.
Supertypes are a honking great idea. We should use more of them.

This massively cleans up the node types, without polluting the AST with
`expression` nodes.
Gets rid of a bunch of ad-hoc node type unions.
You know the drill. We just make an anonymous node named instead. In
this case, however, we have to be a bit more clever about how to rewrite
it. We turn the sequence of a type followed by an optional ! into a
_choice_ between mere type or type followed by bang (the latter being
our new named node).
Adds a new type `nested_type_identifier`, which contains the
choice-branch that previously allowed those tokens to bleed through into
the closest parent field.
Because `_type` was anonymous, its body was inlined in all of the places
it appeared. Because this body contained a `name` field, this field was
_also_ inlined. This caused a bunch of nodes to have spurious `name`
fields, and for some of them (that already had such a field) it caused
that field have multiplicity greater than one.

To fix this, we make the `_type` node named, which prevents the errant
field from escaping.
Same pattern we've seen many times before: a field on an anonymous node
gets attached to the parent node instead.

I'm not 100% sure this is the right solution, but it seemed wrong to
just make `_parenthesized_type` named instead (we don't usually name
parentheticals). At the very least, this cleans up the spurious
navigation_expression.element and tuple_type_item.element fields.
Same procedure as before -- we change the anonymous node to a named
node, and the problem magically goes away.
Hides a bunch of huge unions under (hopefully) sensible supertypes.
@tausbn tausbn force-pushed the tausbn/unified-swift-grammar-cleanup-phase-1 branch from 7c91dd9 to 911e59c Compare May 12, 2026 12:58
@tausbn tausbn marked this pull request as ready for review May 12, 2026 13:17
@tausbn tausbn requested a review from a team as a code owner May 12, 2026 13:17
@tausbn tausbn requested review from asgerf and Copilot May 12, 2026 13:17
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR continues the unified Swift grammar cleanup by introducing explicit supertypes and normalizing several grammar rules (types, expressions, conditions), then updating the unified Swift dbscheme and QL AST library to match the regenerated tree-sitter schema. It also adds tooling and documentation to make future grammar edits easier to regenerate and review.

Changes:

  • Add unified/scripts/regenerate-grammar.sh and document the Swift grammar regeneration workflow in unified/AGENTS.md.
  • Refactor unified/extractor/tree-sitter-swift/grammar.js to introduce supertypes and make key constructs (e.g. expression, type, if_condition) first-class named rules.
  • Regenerate/adjust Swift schema surfaces: unified/ql/lib/unified.dbscheme, unified/ql/lib/codeql/unified/Ast.qll, and add unified/extractor/tree-sitter-swift/node-types.yml.
Show a summary per file
File Description
unified/scripts/regenerate-grammar.sh Adds a one-shot script to regenerate tree-sitter outputs and refresh node-types.yml.
unified/ql/lib/unified.dbscheme Updates Swift dbscheme to reflect the new/cleaned grammar structure (including new supertypes and field normalization).
unified/ql/lib/codeql/unified/Ast.qll Aligns the Swift AST QL wrappers with the updated Swift dbscheme/schema.
unified/extractor/tree-sitter-swift/node-types.yml Adds a human-readable node-types view for reviewing grammar changes.
unified/extractor/tree-sitter-swift/grammar.js Introduces supertypes and refactors rules to reduce ambiguity and standardize node shapes.
unified/AGENTS.md Documents how to regenerate and review Swift grammar changes.

Copilot's findings

  • Files reviewed: 5/6 changed files
  • Comments generated: 1

- ">"
- ">="
- ">>"
- "?"
Copy link
Copy Markdown
Contributor

@asgerf asgerf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work! 👍

@tausbn tausbn merged commit 5508b15 into main May 12, 2026
14 of 15 checks passed
@tausbn tausbn deleted the tausbn/unified-swift-grammar-cleanup-phase-1 branch May 12, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation no-change-note-required This PR does not need a change note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants